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THE  INFLUENCE  OF  ENVIRONMENTAL  NON-STATIONARITY 
IN  A  SEQUENTIAL  DECISIOM-HAEINO  EXPERIMENT 


M«rrlll  Mo  Flood 


Sunnary :  Th-li  papT  reports  >1on  a  •erlas  of 
pilot  axperljaenta,,  and  on  their  theoretical 
bac kg ro und ,  t t ■fee  <^wiy  the 
effect  on  human  declalon-nvakln^  of  a  belief 
that  the  environment  la  changing  when  In 
leallty  It  la  constant.  The  results  suggest 
that  subjects  tend  to  aearch  more  among  poorer 
altematlvea  when  they  believe  that  the 
situation  la  changeable,  and  In  conformity 
with  mathematical  models  suggested  by  W.K.  Estes 
and  Ro  R.  Bush  to  describe  the  two  types  of 
declsiorwnaklng  boliavlor,  ^  \ 


1.  Introduction. 

/ 

V.  K.  E8t«8  reported  certialn  experimental  results.  In  e 
talk  at  one  of  the  early  plenary  sesslonB  of  the  Seminar,  tlwt 
etlmulatcd  me  both  to  review  soaie  old  data  of  mine  and  to  con — 
duct  a  new  experiment  ualng  human  subjects  In  a  multiple- 
choice  situation  provided  by  the  punchboard  demonstrated  during 
my  own  plenary  session  talk.  Ttoe  type  of  result  .obtained  by 
Estes,  that  Intrigued  ise,  has  been  reproduced  by  other  Investi¬ 
gators  In  other  laboratories  both  with  human  subjects  and  with 
rats;  It  Is  as  foUowG:  In  certain  2— choice  situations  where 
the  reward  probabilities  are  ft  and  Tt  the  organism  will  tend 
eventually  to  choose  the  1  alternative  with  probability 

^  p  - - -  Where  1  f  j. 

For  example ,  Estes  reported  a  case  In  which  the  rewa^  proba- 
bllltlh'B  were  Tj-1/2  and  Xa»0  and  the  average  tendency  of 
ricveral  human  subjects  was  found  experimentally  to  be  p'5^.67. 
Irr  Mental ly,  Eatea  remarked  that  the  formula  for  not  only 
scribed  all  hla  experimental  data  well,  when  averag^  over 
enough  subjects  and  trials,  but  that  It  was  derived  from  a  sys¬ 
tem  of  mathematical  axioms  forming  a  basis  for  his  learning 
theory. 


During  the  dl8cus«lor  of  the  Estes  paper  It  wao  argued 
(incorrectly)  by  sone  of  the  theorists  in  the  audience 

tJiat  such  behavior  was  surprisingly  "Irrational"  since  the 
optimal  strategy  for  tt»©  crganisa  is  clearly  pure  rather  than 
mixed,  and  so  the  organlaia  should  leam  eventually  to  choose 
or.ly  the  alternative  the t  provides  the  aore  frequent  reward. 

I  countered  with  the  following  two  objections  to  this  game— 
theoretical  argument r 


(a)  The  payoff  utllJtles  could  not  be  well  defined,  as 
Is  so  unfortunately  the  usual  case  In  any  of  our 
attempts  to  apply  game -the ore tic  arguments  to  a  real 
case,  and  t^>ere  la  a  reasonable  payoff  matrix  that 
would  rationalize  the  reported  behavior.  Thus,  If 
the  organism *8  object  were  to  get  the  very  best 
score  that  It  could,  rather  than  simply  to  maxi¬ 
mize  Its  expectation,  then  It  should  sooetlmea 

not  tend  to  uae  a  pure  strategy.  I  Illustrated 
this  case  by  one  of  my  punchboard  experiments  In 
which  a  very  Intelligent  subject  remarked  that 
ner  only  hope  to  get  a  "perfect"  score  In  a 
(90,10]  rxperlment  would  be  to  play  90  In  the 
90-ca8e  and  10  In  the  10— case  and  be  lucky  on 
»ach  play;  In  other*  wor'ds,  If  absolute  perfection 
wore  asslgiod  sufficiently  high  utility  her 
optimal  lehavlor  oven  In  the  gnrrie— theoretic  sense 
would  not  be  to  always  choose  the  oome  case.  Of 
course^  In  a  (50,C)  experiment  this  same  argument 
might  suggest  that  she  should  use  a  single -choice 
strategy  but  only  If  she  Is  thoroughly  convinced 
that  one  response  will  always  be  non— rewarded . 
kt  any  rate,  a  weak  defense  of  tl.e  mixed— choice 
behavior  can  be  made  along  these  general  lines. 

(b)  The  von  Neumann-^lorgenstem  game  theory  Is  not 
applicable  In  this  situation  unless  the  oi'ganlsm 
can  safely  assume  that  the  experimental  stimulus 
is  generated  by  a  stationary  stochastic  process. 

?or  example,  if  the  organism  believes  th^t  there 
mny  be  senw-  pattern  {non— 0  tatlonarlty )  over  time 
In  the  stimulus,  then  It  con  often  do  better  by 
using  a  mixed  ratlier  than  a  pure  stretegy  for 
otherwise  it  would  nave  no  way  to  dlscovei’  any 
oattrrn  eff'^ct  after  the  I  Imo  tt\a  t  It  settled  on 
o  pure  strategy.  In  fact,  thr  present  state  of 


mathenat  lrai  g«mr  th<>ory  (and  btatlBclcal 
«^^*ory)  i's  auch  thsVthe'r^?  le  no 
goripralTy  accepted  prcflcrlptloh  of  optimal 
behtivloj-  in  the  nerj-^fca tionary!  coae ;  It  may 
veiy  veil  be  that  organlotric  behavior  will  be 
fovind  to  represent  euch  a  flo3ut.lon  If  and  when 
we  underot  and  It.  In  reoponsej  to  my  query  on 
this  po^ot,  Eeteb  paraphrased  the  Inrtruc tlons 
giver:  to  hlo  8ub;.?ctB  and  it  aeemed  .ilgnlflcant 
to  me  he  had  made  no  effort  ta  suggest  to 

them  t.h&t  they  were  in  fact  being  confronted 
with  fi  ptatlonnry  process.  In  my' owii  cKpcrl- 
ments,  on  the  other  hand,  I  hod  crmphaslr-od 
very  strongjy  the  eKoct  nature  of  the  station-* 
ary  process  confronting  toe  8utJ‘>r‘t8^  and  It 
had  sreB'cd  to  me  that  th?  c  fact  haJ  helped  the 
Mubjrctf  go  to  pur'--  8tr»i«eglca.  I  “ on  ,U  e  tur«d  , 
there  t'nat  th>»  pur-e  type  of  h^dif.vlor  would 

O''  founh  In  nubjerta  wh.o  worb  convlrccct  of 
i  ut  lonr,  r1  t;y  and  th*'  mixed  type  of  hehn - 

""viof  wo\. Id  nometlmco  hr  found  In  subjects  who 
ix'llfved  there  might  be  non-  a  »-a  tl  ona  rl  ty  . 

The:>'-  it  sues  hc'"'  beop  irn.co  dJ  rcusned  during  the 
G'-mir.ar  and  ifu  aec  nrn  to  be  Bgropnrnt  thr.t 
th'-'  rrtfjtle)-  in  vo' .  -)y  of  f'jrthei’  * -’perlmenta  1 
i  rn/o.s  tlni  r.l  on . 

H .  :<  K.ph  offered  i  v t ! '-a  !  o,'<pia’\aLlc:Ti  of  thr-  pure 
and  mlxe-l  at  Ifie  ini  iJ 'll  fr''''-tlrg  of  ore  of  to/-  four  work 

!  ng  groups  firtt  fc  irr.rd  wl  ;  uln  ih*'  S'c.b*/.'  h  cv-p'd  ihie  cn 
'•'■ie  n t'lc  lus t  1 If  firnlrg  mo'-l'l  pro,-or^ed  rrr*’T.t'y  by  Duah.  and 
:  -ell.  Bueb  lid  this  by  nnowleg  t.ow  tj-.f.  Kates  fo’mnula 
i ob  .ol  by  assigning  >■;  t/  li^  .‘-ry  !:;1  values  to  the 

;  te  s  In  th''  fcjunh- Mo*' ;  J mt  t-* t.i.o'na^  It  a  t  model,  as  later 

wrl ’t'r.  '’.own  in  the  first.  V'iHKING  of  the  Seminar  (WM— l). 

3ueh  .i  i  t’  derived  an  .t  lympt.  .•)t '  m.i  xturo  fomula  for  the  r-chol 
case  -tncl ,  wllh  (1.  L.  ';hompr;-')n  I'WM  8),  ro.’n/' rbed  on  some  of  the 


fchr'-.’.  t  lO'igh  1  otr''>e.''ert  tlil  i  po ! 
s.  -•.'•{/t'  i  tiy  two  of  my  col  I  ■  -  gu'- 
'•,c{>-rl 'ir  Ti*.  .  rriao»>  iT-'Tor.-  v  -s  ^ ro 
t '!'■  wrMt-*r;  1  n?  true  t  1  t'n:*  ^r;.'  fr 
1UU--S  o'<l  fnl’-'-.r  -t  *;  -i-  '  . 

I  ;  >07  r  1  «  -.1 

:  .  .  ’  y  i  ■  L  '  •  '••I. 


>7  grent'y  it  w&s  not  really 
i  ..bo,  nub  Joe  ts  In  one 

t  h:--  1  *1  I  us  t  re  five  example  In 
..m  .tpu'iernt  t",  Itfrnn  eu'- 
-I  O'  :  o  bus  till  on* 

,  .  ’  f  ■  *  b  ve  re 
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CKp^rJmental  questions  Involvad;  the  r-^iholee  formula  of 
Bush  Is: 

_ ^ _ 

«i - for  i  “  1,  J?,  r. 

r  -i_ 

i-'j 

T.i-f  punchboanJ  rxp^r’lmer’to  Jvioiude  InstancoB  in  which  and 

r»^,  so  they  provide  some  dnt/i  thnt  are  relevant  to  the  question 
c^rceml r.jc^  the  applicability  of  the  pure  and  mixed  models  dis¬ 
cussed  by  Bush. 

Mosteller,  In  one  of  the  Seminar  meetings,  presented  some 
experimental  results  the  2~<:holce  case  taken  from 

TN'cent  work  of  St-inlry  with  rats  In  mazes  and  from  Jerrett 
with  humans  done  with  a  *'two..#inn  bandit"  constructed  at  the 
lii.^tlgatlon  cf  3unh  and  Mo9‘eller.  He  dlo:uaaed  the  quality 
of  agreement  between  these  experimental  data  and  the  pure 
and  mixed  models  nnd,  -imong  other  things,  coriclud'»d  that  rats 
and  humans  both  seem  tc  gee  to  a  choice  of  the  better  alterna¬ 
tive  moTv  rapidly  when  they  are  rewarded  less. 

In  thl3  rnemc  ••ariduM ,  1  nhiall  dlocusn  some  punchboard  cxporl- 
rn^nta  pji.'-  in  the  p-oholcr  and  9-<*holcr  casea.  In  particular, 

I  Ghalj  present  the  reoultn  of  a  pilot  experiment  done  aa  « 
T'-Gult  of  th»-'  Eotea  talk  In  an  effort  to  obtal.i  either  pure  or 
mixed  behavior  by  human  sub.lects  according  as  they  are  or  are 
not  ronvlncrd  ol’  t,i. . .-.s tat  1  r tty  '!r.  a  i  ane  that  is  actually 


ala t Iona ry. 
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2 .  The  experlaental  nethod.  T 

The  equipment  and  procedures  used  aro  explained  In  the 
written  fnstructlons  given  to  the  subject  before  the  trials 
start.  The  set  used  In  the  Initial  9~cholce  experiment,  where 
statlonartty  was  stressed,  are  Included  here  as  Appendix  A. 

The  set  used  ir  the  2-<5holce  eTporlinent >  where  statlonarlty  was 
not  mentioned.  Is  Included  in  Appendix  B.  I  refer  the  reader 
to  these  two  Appendices  for  information  about  the  equipment  and 
pi^edure  used,  and  now  assume  familiarity  with  this  material. 

In  (A),  the  9— choice  case,  the  ten  sets  of  reward  probabi¬ 
lities  were  each  chosen  at  random  on  the  Interval  (0,1).  In 
(B),  the  2— choice  cane,  the  ten  pairs  of  reward  probabilities 
were  chosen  arbitrarily  so  as  to  provide  Information  on  pairs 
Bcattei-ed  rather  evenly  over  the  possible  range;  the  equipment 
was  Identical  with  that  In  the  9— choice  case,  end  even  the  same 
codes  were  used,  but  the  subject  wan  limited  to  cholcas  between 

a  pair  of  columns  specified  at  the  top  of  each  code.  The  actuajl 

! 

values  for  these  pairs  are  as  follDws: 


Game 

1 

2 

rr 

4 

5 

6 

7 

8 

— — i 

9 

10 

TOO 

TT  i 

51 

76 

55 

40 

80 

56 

98 

■^9 

92 

too 

TTa 

24 

09 

09 

0 

30 

44 

49 

59 

69 

17 

In  >),  thfre  were  ‘  subjects  and  each  did  the  same  ten 
punchboaids  wit  i  ’00  choices  on  each;  one  of  the  two  subjects 
had  pr-evlously  done  p\inrhboard  following  the  Instructions 

of  i’protdlx  A;  th'y  were  both  qtilte  urlfamillar  with  game— 


rOicor'c t,lc  notions. 


D‘tta  Jiio  -r.; 

t’:ar?  interpreted  wltn  't  r.  io 

.  jf ,  ■• 

tai.t  ,t iMi/jriC-" .  oui*  ch.’ert,  Ir-i 

^UW'’!n  nubjo.-’  tr.  bohnv.'?  or  rere  '>f  ■ 

:t  ' 

•  I 

relo-in*?;!  thii'i;— t tiey  )wliev>-  i.r  V,.* 

''  ':.C'  -)  to  t h'  P’lr.  ■’  ■  ■ 

>'r  c  •  nof.oi  t'h-j't  j  ■ 


. .  cjccj  pi;  -t-  tbey 

r:  ;:r?  we il-iSe fined  hypotbeola.  *  ^ 
tn  r,..et  th«^  gene^THl  notion  that' 
:  e  IK  ?  red  niodel  I'or  reaj  enj<a 
•  thei'  tcb.ro/e 

.  .  •"  ''o  L  leve  thv  t  tl;.  y  r.cve 
•‘ij)ecif ca  ]  .1  y y  o'-  rre- 


■r,'-'  •  .:  r  t 

/.io.i,  Hi  hve 

f.-'oqMC.'i  ; '  • : 

'• 

•ho  ceC,  in  vu:A'  'T^-rv; 

err-.'  cir 

'JL  -'it  ■  -  .  .  '/  ■ 

{tot  if  rrr.. 

> 

r.  <-•  pif'd.'i  t.  .1 

.  .  j 

*  i'  •.  .  Jk  J  , 

ny  rv:  out<  u- 

rhinli  rormi't:"'. 

-  •  .(  • 

.  C)  e  /r.  i-:. . 

'.',Vr”‘e  :■  / 

.1  Vv.a 

’  '  -v  big  v/nr :  ‘  j ^ 

j'nhoo  id  TT't-?r.i  i 

r  ‘.n.lr  eoriv 

t 

:  ..r.,  /  f  V  1 

1  i  e 

li*' 

loJlci-'A  /no 

.  T  ■■  ' 

>  i-v-!'  ,i'j  <1  ./vcr  Lir;'-?.;! 

1  f  ..’Vb— 

iject  o'iourui  !;/:ow  v<;r;.'  Ojf''  i  o, 
early  i  lais  oct’orfjln;: 

t-  • 

»o  unity  on  the  other,  ioowu 
tSubJ-^ctD  rruiy  behavr  riocojdip' 

W- 

period  .at  ’r.o  boglnni.t'jg  :•"• 
enrly  rxeo.  n- i  m*  load  a  thet-i  to 
In  I'  lel,  atat.lonary :  Jnutr  i  .  11 
conn  hi;;/’  In  iAie  preamt  Oii'' Je  1  • . 
..  It  might  be  possible  to  .r,.  -/; 
confirming  the  oub.j^ot’r  ri-.j.v-i 
tlor!..r1ty.  perhaps  c/.m  by  ,  • 

lo  the  danfcr  In  thJo,  of  «:orr.;  . 


TO  ;■  ••  over  r  ^  ^  nga  fiptiipJe 
:  r,  r  -  on  tnt?  nna  betT? ,  c  r  cloo:.* 
'  •  ,  i '.  Is  qu  ite  pons  J  u  te  that 
jn'.x.'id  CiCKlel  for  o  cci-toln 
i  to  v)ic  piirtr  model  nfler  chnii’ 
h.i  bolinf  that  the  preer  rn  in 
:  I'.'j.o  iiypotln-nle  thtt  1  rb.il 

'.  rnei-ratlons  more  dlroc  tl./ 

'  ■..-ilicr  v/Jth  I’CCS.l-d  to  f;tP- 
■  ■  -'n  r ter  each  trie  1 .  There 
T.Mt  .'.uch  quc'VJA-o  v.ight  alter. 
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th.  ^experimental  stiimiluH  and  thus  affect  his  responses, 
of  this  danger  rould  be  avoided  by  asking  all  questions  at  tn.- 
oIos..  of  the  work  by  each  subject,  and  I  ha«e  aometlmes  dore 
Ldlfi.  1  have  nothing  ayatematlc  to  offer  as  yet  on  this  ques- 
I  '.cn,  except  ae  som.f  Inferences  may  be  made  on  the  basis  of  trje 
t  'U:3  data  itself.  In  T^ture  experiments,  I  hope  to  approach 
t  3  'jueat.ion  mor*-  directly. 


of  the  .  ri  nr  Ipa  I  data  fer  the  ^-choice  case  ore  fi'e 
Sf!  tod  In  rnt  loB  1  inc  rcr*  Subjects  PD  and  MS;  ti-.f-se  date 
wor»'  tfi  ini*r  t.-sl.'g  the  Int  l run t lone  of  Appendix  E.  Analogous 
datfj  f-^r  the  1  /ire  prcar-Titoj  for  Subjects  1-10  in 

i thea^^  data  wore  obi.ulned  using  the  1  ^..-^tr-jc tl oro 
of  /'open.  I  lx  »  . 


TABI£  1 

?-~Choic»  (Sut> ject  PD) 


I  Colmn 


I  _ 

40  100 


»  — rzzjs -  ;  fj  ••  ir>erc^ntage  of  time  the  tti  colunui  »»»« 

choeen  during  trials  x  thi y; 

Cf  «  trial  nuisber  when  Tj  o.iuan  .j  pt 
_ _  choKfn. 

L  means  that  3.  cri.rvi  v.-ij  c  ■>  ti  *?  ‘'efi;  wr.d  ’  U)At  l  *r;i3  r,r 
the  ri,;ht  otf  tie  i«  ccluran. 


Thl.'i  in  the  orcei  o**  pUv  r<'-  In  ocJ  » 


TABLE  ^ 


LC-*  (I 


Cod#  Coluan  i  Or<l<»r 


^  i 


8 

i 


._L. 


_ 

€)4 

96 

I  91 

I  100 

i  ^ 

I 

f  —  — 

: 

I  98 

I  oa 

I 

1  QV 

I 

i 


;  c*  *  ' 

- 1^: - 1 - i - 

I  ^7  I  00  j  8?  57  I  >16 

i  55  I  8l  \  ti  69  ^  •: 

;  41  I  68  ,  65  i  91  , 

i  100  1  97  I  0  •>  \  'X- 

I  55  i  5j  ■  42  8^>  ‘  r. ; 

■  I  ! 

i  29  j  04  J  1 00  C 

•  56  ;  07  !  0  I  5  ;  '  •''' 

:  ;  100  ;  0  1  '  •: 

=.7  !  46  \  ^7  74  68 

!  ■  ' 

>  97  i  0  ^  ’OC 


74 


1  ^E_  4 

v!-:.v  '  .ouv.,cr 


Col'j«n  i  Orirr 


5  i  5 
(>  1  6 


I 

!  6 


T.;# 

I 


.  .  8 

.  f  1  ^ 

-JiLL 

c 

.IO0I 

c-.  57 

t'l 

hO 

■  00  ! 

■J*  ’  65 

86 

yo 

82  j 

-  ;  "^1 

O.’ 

9^' 

; 

I'X 

fVy 

03 

i 

r 

‘  K'.o  i 

• 

0 

. 

t.:  ' 

i4i> 

‘C 

'• '  0 

vH  -^.y 

( 

f 

\ 

.0  j 

.  65 

K’C 

y 

1  ■ 

;  1 K  f 

i 

■  6/ 

ion 

1  ; 

.... 

<8  ■ 

.* 

:) 

t 

column 

for  whl) 

h 

'jbAt  1 

1 1  '  7. 

1  A  V  ,  «  J 

'  th<»  8*JbJcct  p; 

^  Vt  *1  t.li« 

•  ^  p 

ot!<  n . 

rui  .:  f''r  v; 

il-'h  r>^  . 

H9  frm 

.  -M  . 

•Ti!  1ji 

'  *1  -rr 

\  d  X  /  -  *  « 

1 

T  iW,  ci‘ 
o^'  o;.*  ri  iHj  "  . ', 

tf  c  . 
‘A.y 

)l»ir!i  «'•! 

■  1  r 

»  rt.- •  n*  ..ti?: 

i  ►  1 '.  h 

T-J 

T  r.O’-  f  !iO*»#‘ra 


TABLE  5 

9-^hoier  (Subject  3) 


It  MttMi  V9VJ  iinlllmly  that  tte  ehoioM  of  Sub  Joe  t  PD  con 
bo  •xplolno^  Mil  In  terw  of  the  rlzed  aodel.  for  example, 
on  hie  leat  board  all  hie  eholeee  mere  made  on  a  column  that  gave 
mine  on  only  56  per  cent  of  the  triale.  One  poeeible  explanation 
of  thia  behavior  le  that  he  Mt  a  etandard  of  30  per  cent  vine  as 
eaiiefaetorjr  and  then  did  not  search  for  a  better  column  as 
long  as  he  felt  that  this  standard  was  met,  aa  mat  the  caee  on 
this  last  board  after  the  sixth  trial.  A  fact  conaiatent  with 
this  explanation  le  hie  continued  effort  (t2  attempts  through 
trial  86)  on  hla  first  board  to  vin  on  a  coluam  that  never  paid 
off  mhlle  the  better  column  mat  paying  off  lees  than  half  of  the 
time.  These  poealbilities  are  typical  of  the  experimental  coov- 
plioatlons  that  arise  becauae  of  tne  particular  psychological 
set  that  comes  with  the  subject  at  the  outset. 

Sutject  NS,  on  the  other  hand,  seems  to  have  performed  In 
a  manner  quite  consistent  with  the  mixed  model,  ea  judged  by 

aO  1,100 

the  close  agreement  between  Pi  end  ft  ;  his  eeventh  board  Is 
a  notebl«  exception  nnd  It  Is  a  aurprlaing  fact,  In  this  con— 
nertioi.,  >hat  he  devoted  hi')  last  ?8  trlela  to  a  choice  that 
alvay^J  failed  and  that  hud  fllrfaya  fell^^  or  hli-  prevloua  39  trial 
with  it.  It  is  algr.ifl'ant  that  he  Jeept  trying  both  column  vel. 
through  all  tvo  of  his  boards,  ar.d  that  he  remarked  durlrg 
and  after  the  experiment  that  Ue  v^s  convinced  that  there  ves 
some  sort  of  pattern  and  thnt  he  might  find  It  If  lie  kept  ci 
luffitins  for  such  rei^laritiea.  If  we  euppoee  that  this  hurUng 
ended  eoon  alter  trial  ca  t-.cn  me  would  ex.-ect  that  the  f]  ^* 
might  bip  In  even  tetter  ajreean  nt  with  the  than  ar»*  tl*- 


flflOO,  Mtw  to  bo  <|ult«  tivo  for  Siabjoct  MB,  oseopt 
for  hla  ninth  board,  and  la  fbrthor  avtdcnea  of  oonPlatoney 
bttwaen  hlo  bahivlor  and  tha  notion  that  tht  aljnd  nodal  appliao 
until  tha  aubjaet  ballataa  that  tha  prooaaa  la  atatleaaay. 

Tha  data  on  Subjaata  l-»tO  ara  ratter  hard  to  Intarprat  In 

light  of  our  cantm  hjpothaala.  Cartainly,  thara  la  no  vary 

atriklns  agraanant  bataaan  jMMid  It  doaa  aaan  to  ba 

tha  us'jal  oaaa  that  tha  aubjaet  la  atill  ebooaing  eoluBie*  otter 

than  tha  boat  ona  through  nost  of  tha  100  trial#  in  aaah  gate, 

and  th^  aubjaeta  uaually  fall  to  aattla  down  on  tha  aary  beat 

0- 

jhoire  within  thalr  hundred  triala.  Of  eourea^  it  naj  raeeon— 
ably  ba  nrguad  that  tha  aarly  trlala  In  aaeh  gate  ahould  not 
ba  taican  too  aarioualy  in  an  invaatigation  of  aeyap^^l*  bate 
vior;  for  thin  raaaon,  1  hava  inelndad  tha  f^^'^^deta  but  tte 
intarpratatl  >«'•  la  difficult  hare,  too. 

Ona  aourca  of  trouble  in  tha  analyaia  la  tha  alallarlty 
batwaen  the  in  e^  eaeea,  so  that  it  la  hard  for  tha  aob-> 
Jaeta  to  dlseriwinata  batv^en  such  eoliaans.  for  axnapla,  tn 
Code  7  the  three  largest  ara  96,  95t  and  90  to  that  it 
would  not  be  at  all  aurprlalng  if  a  subject  were  to  apand  aowa 
of  hla  ohoioes  on  the  93  par  cant  colvan  that  tha  uIjnnS  attdel 
would  alloeata  to  tte  96  per  cant  and  90  par  cant  eolaun.  Con» 

I 

ieqvioM tl/,  in  an  effort  to  Hoften  this  confuting  affect,  Xim 
data  werp  ragroupad  for  anelyala  by  eouiblning  coluMit  with  etib- 
parable  values.  Tte  ml«  for  this  regrouping  was  quite 
arbitrary .  It  ecnaiatad  Ir  ocaibining  all  data  for  those  eoltete 


viioo^!  T|  iralu«a  differed  by  7  per  cent  or  less  froB  the  oyi? 

}iito  n  new  eoluan  1,  tlien  slsillarly  forming  «  new  column  2  to 

All  thmie  coltvimii  whoee  veluet  differed  by  7  per  cent 
or  leee  fron  the  mtx  not  already  included  In  the  new  column  i 
a;  t  continuing  until  all  the  eolumna  were  regrouped.  If  we  let 
dr '.otr  the  number  of  new  eoluwia  formed  for  code  J,  and  let 
»j!.  for  denote  the  mean  of  the  set  of  values 

reprt^Aont.ed  in  the  new  column  k  of  code  J,  then  the  mixed  model 
K*<{ut''ea  that  the  eaya^totic  frequencies  of  play  on  new  column  k 
for  j  ere  given  by  the  following  expressions  for  p^j 


for  10;  r^. 


J  ■  .1  ■  y 

dia 


The  TV  grouped  dets  are  presented  nnd  compared  with  the  values 
•0  Ob  ,r/ined  .'cr  in  Tnbl^n 


Subject  9  ««e  e*clui*-d  In  nowputln'j  the  •veraces  beceuae  hl« 
erlectlona  differed  »o  f-xatly  frr  the  other*. 

f?  l8  the  percentftg*  of  tli^  that  couw  Ir  vas  choaen  during 
trlala  x  througli  100. 


TABLE  ?0 


Subjpctj  fl  I  f i  I  fi  1  fi 


7  100  " 

8  100 


Average  1  96.0 


-Choice 

f  (Coe 

LSL 

10 

9 

0 

0 

o 

c 

0 

2 

0 

0 

0 

0 

0 

0 

0 

5 

1 . 4 

2.0 

6 

5 

^  t.-' 


It  would  a  atrotoh  on  th«  Imagination  to  argu«  that 
Taolta  provi4«  avldanoe  in  support  of  tha  hypotheala  that 

3ur  subjects  bshaved  according  to  the  mixed  iKXlel,  but  a  otlii 
greater  stretch  to  argue  that  these  data  permit  us  to  reject  the 
hypothesis..  My  own  xlew  is  that  a  jwe^i  more  extensive  exxwriiaer  t 
and-m  better-eonducted  one.  Is  neoeaaary  before  any  real  oooolu- 
Blnr.a  oan  be  drawn.  I  believe  that  the  aioat  pronlsing  approach 
le  to  start  with  the  >-oholee  ease  In  which  the  t*  Are  rather 
evenly  spaced,  and  not  too  close  to  0  or  1 ,  In  order  to  oheox 
the  closeness  of  agreec'.ent  with  the  Bstes-Bush  formulas  for  p^. 

Wot  u/itll  experimental  teohnlques  are  good  enou^fj  to  give  constant 
and  repeatable  results  in  SKr.  Moojt  for  the  '>-choloe  case  with 
either  the  mixed  or  pu/v  modelo,  at  will,  would  I  want  to  tackle 
the  problem  of  datermlnl  ig  the  precise  nature  of  the  experimental 
stimulus  that  lb  neo<  csary  to  produce  this  kind  of  difference  in 
behavlor-'-and  only  after  1  undez^tood  this  stimulus  problem 
reasonably  well  would  I  again  want  Lo  Wurk  with  the  r-^oholce  case 
for  r  >  3.  The  onjde  pilot  experlr  nta  reported  here  may  be  of 
Bom*  use  to  others  participating  In  the  Seminar  who  have  beooew 
interested  in  this  problem  of  the  pure  vs.  mixed  learuing 
t.*havlor. 


kfnmn  k 


maywycTioMS  vm  srmc-^iim  EmnrmK 

"  ‘  ^ 

t)i«  equipaMmt  for  •tatio-oiao  oonaiats  of  a  ”pinoM»oiPii»*  a 
’'punch,*  c  ’’ooCa,**  and  a  "raglatar.*  Hm  coda  and  ragislMP  art 

I 

pr:nted  foma  faatanad  to  tdia  back  and  the  front,  raapaatlvaiy«  of 

/ 

the  punchboard.  fha  puiiSh  la  uaad  on  each  nova  to  aatoa  a  bolt  la 
^hc  raglstcr  aianlfylag  tba  choice  of  an  Intagar  frea  1  ti»an(^  9 
Samples  of  code  and  regiatcr  arc  attached  to  Uteae  Inatruatlona* 

The  register  has  23  llnca,  and  !n  each  line  the  Integra  1 
throjch  9  appear  in  four  "fielda*  ac^ose  the  page;  thia  provldaa 
l  or  100  fnovea. 

The  code  haa  either  a  1  or  a  0  in  each  position.  A  1  daaotca 
a  tolu  and  a  0  denotea  a  loss.  The  c  >de  is  arrange^:  ao  that  tha 
n>ark  1  appears  a  preatalgned  nunber  of  tinea  In  aaeh  eolunn,  Ttie 
order  of  tr«e  narks  1  and  0  in  each  eoluan  la  randon* 

Tne  fi,^3t  move  is  nade  by  punching  out  one  of  the  nine  digits 
r>v.  1  of  field  1.  If  a  1  Is  seen  thro^jgh  the  hole  this  position 
J8  circled  In  pencil  by  the  player  tc  denote  a  win,  otherwise  it  is 
left  unclrcled  to  denote  c  loss.  The  second  nto/e  is  Mde  aimllarly 
by  punching  in  row  3  of  field  1,  eni  circling  to  denote  a  I  if 
observed..  Aftpr  the  23  novea  in  flexd  1  are  conpleted,  start  at 
the  top  of  field  2,  and  continue  in  thia  way  until  all  tOO  rows 
have  been  pvmched.  The  s  re  on  these  100  sMves  :s  the  total 
number  of  cirelad  positions. 


After  you  have  made  100  trials,  you  will  give  the  ’■■a 
nniructlons  for  your  plays  In  the  next  hundred  trials.  Yo':  do 
h  13  by  assigning  to  each  choice,  1  through  9>  a  number  indlcatlrig 
c-iv  r:ften  you  wish  that  choice  played  In  the  next  100  trials.  Tr.r 
nine  numbers  must  add  to  100.  For  example,  you  might  assign  i?0, 

5?,  1?,  10,  25.  to  digits  1,  3,  7,  8,  9  respectively,  and  zeros 
th"j  otners,  Indicating  that  you  want  digit  1  played  ?0  tlmea,  dl^.H 
?  played  no  tinea,  digit  played  32  times,  etc.  In  particular. 

If  you  wish  to  have  some  one  number  (say  5)  played  all  the  time, 
_hcn  you  would  assign  100  to  that  one  and  zero  to  each  of  the 
otriers.  These  numbers  should  be  written  in  the  ro*:-  provided  on 
the  data  form  for  this  purpose- 

Your  object  Is  to  get  as  .many  wins  as  you  can  in  the  200  moves 
Informat’on 

This  Is  an  experiment  designed  to  obtain  a  quantitative 
oorparlson  of  the  ability  of  peopl^.  with  that  of  rats  In  learning 
1.0  piay  a  certain  simple  Intellect  al  game.  Bush  and  Hosteller, 
s-t  Harvard,  have  examined  experlmeijta J  data  obtained  by  psychologist 
In  t.helr  studies  of  rat  learnir;g.  they  have  developed  a  mathe - 
mat:  :jl  model  thf.t  seems  t.u  fit  t.n*  rat  data  quite  well.  I  am 
using  this  model,  wh.' oh  they  n.-ive  called  the  "stat  ral,"  to  compute 
the  probable  perfomance  of  rsi.;  l.i  placing  vin  Keumann-Morgorst f..  ^ 
games.  The  scientific  purooeo  le  tc  te-,t  tie  validity  of  varlo  «. 
fi;athematlcal  theories  of  leatrilng  .xnO  decision, 


Tbt^onchbotrd  rtprvit^nts  «  ik  4!)IuiI€A2  wnplrt  ditiMlvii 
and  ^Mi€9  In  thn  follonflng  waOMr.  BM  ««pir«  fiMt  rtiOTgfg 
^  i<  nuiPb«m  bctvdna  0  Md  1  (fdr  «Ktopl«f  *900  or  078)  fmi  « 

'lirn  tabio{  nro  Jioiiglit  of  M  probAblUtlos  3f,  §i, 

y  ^  tout  your  ehoiooo  of  1  throiigli  9  win*  BMb  tiao  you 
r  li  ''  a  n*rbor  fooy  3)  tho  ai|>l.ro  detoffOliMM  wliothtr  or  not  you  Imro 
>r  (  flying  ttm  oorronpondlng  proVnbillty  (0«  Ir  tbio  ooao)  to 
'ic*;  **  t.i«  O^elnlon;  ho  usoo  •  roAdm  oiiMr  toblo  for  ooch  of 

fiecialono.  Biuo»  ii  tho  probability  da  that  your  cboleo  3 
*0  i  /l  .t  wrt  .700,  you  olgict  «xpftet  to  win  7  tfoos  ojt  cf  10  whon 
yo*  ?ifioca«  tno  n^Bbor 

Por  tt'.o  oo<;ond  hundred  triaia  tho  lanplro  oioply  wultiplioo  tho 
rir.e  ntjBb^ra  irrltton  or  ;;oar  data  form  by  tho  oorroaponding  pro- 
v>ablllti«c  Gt,  0a.'“.  -9.  and  adda  thtoo  producta  togothor  to  got 
^cur  tjtal  of  ulna  on  tho  aoeohd  nundrod  trlala^  Of  eoiiroo,  thla 
0;.  >.onax  procodviro/ products  y\»e  aaoo  reault  as  tho  ono  thot 
I  'jv  woui'^  rxj'OCt  tc  got  ir  M'i  actually  through  tho 

•»aconcJ  t.  r.orei  playa  for  you  one  ov  o;.c. 

be  ahoiud  Uko  to  have  you  pia  '  atatlo-nlna  aevoral  tiooa. 
i-'  :r  average  :%iorys  for  all  your  gamer  will  bo  coag/Orod  with  that  of 
toe  :;tat-.'at,  and  tu'je  of  t>H!  other  utjtota  In  thia  experliMnt. 
VO’J  oiu  oe  told  the  nvat  -rat'a  »>*-e  »-ter  ^acn  ^aoe.  xt  tho 
coiif  :£«ion  of  the  .  I  «?end  each  aur.Iect  a  rec-t^'”  • 

Iitjun  of  ^l.e  namea  and  at-jre^  «  f  ».l  playera»  ir.clu.‘!n;  the 

. j . 

l  .ot  K  you  for  j>n  :cin‘»t  • ’g,  y.  1  -V' 
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APPENDIX  B 


fflSTRUCTIONS 

Static-nine 

The  equipment  for  static  nine  consists  of  h  ’’punchboard , " 
a  '‘punch/'  a  "code,"  and  a  "register."  Tne  code  and  register 
orp  printed  forms  fastened  to  the  baeVe  and  the  front,  respec¬ 
tively,  of  the  punchboard.  The  punch  is  used  on  each  move  to 
make  a  hole  In  the  register  signifying  the  choice  of  an  Integci 

<'rom  1  through  9*  Samples  of  code  and  register  are  attached  to 

♦ 

thes."  inatnic tlons. 

The  r*^gl8ter  has  25  lines,  and  In  each  line  the  Integers 
1  through  9  appear  in  four  "fields"  across  the  page;  this  pro¬ 
vides  for  100  moves.  At  the  top  of  each  register  you  will  find 
two  digits  written  In  red;  you  are  to  punch  only  one  or  the 
other  of  these  two  on  each  trial. 

The  code  has  either  a  1  or  a  0  In  each  position.  A  1  denote 
a  win  and  a  0  denotes  a  loss. 

The  first  move  li*  made  by  punching  out  one  of  two  digits 
In  row  1  ■>f  field  1.  If  a  1  is  seen  through  the  hole  this  posi¬ 
tion  Is  circled  Iti  pencil  by  the  player  to  denote  a  win;  other¬ 
wise  It  Is  left  unclrcled  to  denote  a  loss.  The  second  move  Is 
owde  similarly  by  punching  one  of  the  two  digits  In  row  2  of 
field  i,  anj  clrc'lng  to  denote  a  1  if  observed.  After  the  25 
moves  in  fit’ Id  1  pro  completed,  start  at  the  top  of  field  2, 
and  cOTitlnv.>*  In  this  way  until  all  100  rows  have  been  punched. 

Th*^  score  on  these  100  moves  lo  the  total  number  of  circled 
positions . 

Best  Available  Copy 

See_sampl«8  In  Appendix  A. 


Appendix  B 

Your  object  le  to  ipet  es  aeny  wine  ae  you  can  In  the  100 
novee . 

Qeneral  Infonnatton 

This  le  an  experleient  designed  to  obtain  a  quantitative 
conparleon  of  the  ability  of  people  with  that  of  rate  In  learning 
to  play  a  certain  simple  intellectual  game.  Bush  and  Moateller, 
at  Harvard,  have  examined  experimental  data  obtained  by  psycho¬ 
logists  in  their  studies  of  rat  learning.  They  have  developed 
a  Buithematieal  model  that  seetui  to  fit  the  rat  data  quite  well. 

I  SB  using  this  model,  which  they  have  called  the  "stat— rat,** 
to  cosipu^  the  probable  perfomance  of  rats  in  playing  von 

NeuiMUuv-IIorgenstem  games.  The  scientific  purpose  is  to  test 

^  • 

^  * 

the  validity  of  various  mathematical  theories  of  learning  and 
decision. 

We  should  like  to  have  you  play  statlc'-nlne  several  times. 
Your  average  score  for  all  your  gsjoes  will  be  compared  witJx 
that  of  the  stat— rat,  and  those  of  the  other  subjects  in  this 
experiment.  At  the  conclusion  of  the  experiment,  I  will  be  glad 
to  give  you  a  recapitulation  of  the  nai^s  and  scores  of  other 
players,  including  the  stat-rat,  if  you  wish. 

Thank  you  for  participating,  and  g:ood  luck. 
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